Data Loaders

Speech To Text Data Loader

class openspeech.data.audio.data_loader.AudioDataLoader(dataset: torch.utils.data.dataset.Dataset, num_workers: int, batch_sampler: torch.utils.data.sampler.Sampler, **kwargs)

Audio Data Loader

Parameters

• dataset (torch.utils.data.Dataset) – dataset from which to load the data.

• num_workers (int) – how many subprocesses to use for data loading.

• batch_sampler (torch.utils.data.sampler.Sampler) – defines the strategy to draw samples from the dataset.

openspeech.data.audio.data_loader.load_dataset(manifest_file_path: str) → Tuple[list, list]

Provides dictionary of filename and labels.

Parameters

manifest_file_path (str) – evaluation manifest file path.

Returns: target_dict

• target_dict (dict): dictionary of filename and labels

Text Data Loader

class openspeech.data.text.data_loader.TextDataLoader(dataset: torch.utils.data.dataset.Dataset, num_workers: int, batch_sampler: torch.utils.data.sampler.Sampler, **kwargs)

Text Data Loader

Parameters

• dataset (torch.utils.data.Dataset) – dataset from which to load the data.

• num_workers (int) – how many subprocesses to use for data loading.

• batch_sampler (torch.utils.data.sampler.Sampler) – defines the strategy to draw samples from the dataset.

Sampler

class openspeech.data.sampler.BucketingSampler(data_source, batch_size: int = 32, drop_last: bool = False)

Samples batches assuming they are in order of size to batch similarly sized samples together.

Parameters

• data_source (torch.utils.data.Dataset) – dataset to sample from

• batch_size (int) – size of batch

• drop_last (bool) – flat indication whether to drop last batch or not